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ABSTRACT 

This  study  examines  the  role  of  resource  allocation  in  naval 
command  teams.  The  experiment  is  based  on  the  Composite  Warfare 
Commander  -  Distributed  Dynamic  Decisionmaking  (CWC-DDD)  paradigm 
and  investigates  the  impact  of  uncertainty  in  neutral/enemy 
discriminability,  team  information  structure,  and  leader's 
involvement  in  resource  coordination.  Seven  four-member  teams, 
consisting  of  military  officers,  used  the  CWC-DDD  to  combat  a  pre- 
programmed air  threat  to  a  naval  battle  group.  Results  show  that 
increased  average  resource  effectiveness  and  higher  average 
identification  confidence  lead  to  greater  final  team  strength. 
However,  increased  upward  communications  lead  to  lower  final  team 
strength.  Functional  (tactical  vs.  non-tactical)  variations  in 
military  background  have  a  significant  impact  on  team  performance. 
Teams  that  are  predominantly  tactical  develop  more  coherent 
strategies,  more  effectively  utilize  their  resources  and  have 
better  team  performance.  Teams  that  are  non-tactical  develop 
limited  strategies,  assign  lower  confidence  to  their  target 
identifications  and  have  more  upward  communications. 
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INTRODUCTION 


A.   CONTEXT  OF  THE  STUDY 

Large  scale  computer  systems  of  today  have  a  tremendous 

impact  on  the  way  modern  battles  are  fought.  Air  defense 

weapons  can  locate,   identify  and  engage  an  enemy  target 

automatically  without  human  intervention.  Ground  surveillance 

radars  can  locate  a  target,  designate  it  with  a  laser,  and 

send  the  target  description  to  a  fire  direction  computer 

located  miles  to  the  rear  of  the  front  line.    This  fire 

direction  computer  analyzes  the  target,   decides  the  best 

ammunition  mix  to  use  and  picks  the  weapons  to  fire.   The 

message  to  fire  is  sent  to  a  firing  element  located  miles  from 

the  fire  direction  center  and  the  mission  is  fired.  This 

process  can  be  completed  in  seconds  and  can  be  accomplished 

without     human     intervention. 

Despite  this  explosion  of  information,  communication  and 

computer  technology  the  human  remains  the  focal  point  for 

decision  making  in  large  scale  systems   (Kleinman  and 
Serfaty,  1989)  . 

Humans   bring   to     a   system   there   own   ideas   and 

adaptability.  In  many  cases  this   flexibility  to  adapt  to 

changing  situations   is  essential  to  the  systems  proper 

operation.   These  skills  cannot  at  this  time  be  programmed 

into  a  computer  system.   Also  in  many  cases   it  is  not 


politically  feasible  to  place  complete  control  of  our 
country's  weapons  systems  in  the  hands  of  computers. 
Therefore,  human  control  is  an  integral  part  of  weapons 
systems  employed  by  the  United  States . 

The  controversy  surrounding  the  USS  Vincennes  incident 
points  out  one  of  the  problems  that  remain  to  be  solved  by 
systems  designers.  The  Fogarty  report  concluded  that  "The 
AEGIS  combat  system's  performance  was  excellent  -  it 
functioned  as  designed"  (Hill,  198  9) .  The  report  went  on  to 
say  that  "mistakes  made  by  the  Vincennes  CIC  crew  contributed 
to  the  commanding  officers  belief  his  vessel  was  in  danger  of 
attack"  (Hill,  1989)  .  This  conclusion  was  disputed  by  Martin 
Hill  in  an  article  in  San  Diego  Magazine  (Hill,  1989)  .  He 
maintains  that  the  designers  of  the  AEGIS  system  failed  to 
incorporate  enough  human  engineering  in  their  design.  Without 
addressing  the  fault  of  this  incident  it  seems  clear  that 
there  exists  a  need  to  better  understand  the  process  of  human 
decision  making  as  it  interfaces  with  large  information 
systems . 

The  AEGIS  system  is  only  one  example  of  a  large  scale 
military  weapons  system.  These  systems  include  large 
databases,  communications  networks,  automatic  firing  systems, 
several  different  information  gathering  systems  (radar,  sonar, 


observers,  etc)  and  human  decision  makers.  More  importantly 

these   systems  tend  to  have  many  human  decision  makers. 

A  characteristic  feature  of  these  systems  is  the  presence 
of  a  team  of  human  decisionmakers  who  may  be 
geographically  separated,  but  who  must  coordinate  to  share 
their  information,  resources  and  activities  in  order  to 
attain  their  goals  in  what  is  generally  a  dynamic  and 
uncertain  task  environment  (Kleinman  and  Serfaty,  1989)  . 

The  study  of  the  dynamics  of  this  distributed  decision  making 

was  the  purpose  of  the  research  sponsored  by  the  Office  of 

Naval  Research  in  1985. 

B.   RESEARCH  BY  THE  OFFICE  OF  NAVAL  RESEARCH  (ONR) 

The  weapons  of  modern  warfare  are  both  accurate  and 
lethal.  They  can  destroy  cities,  sink  ships  and  wipe  out 
entire  formations  in  seconds.  The  ability  of  modern 
information  systems  places  an  enormous  amount  of  data  about 
potential  targets  for  these  weapons  in  the  hands  of  tactical 
decisionmakers.  Additionally  the  effects  of  these  weapons  can 
be  felt  almost  immediately.  Thus,  although  a  modern  commander 
has  extremely  destructive  weapons  at  his  disposal  and  massive 
amounts  of  information  about  where  to  use  these  weapons, 
employment  decisions  have  to  be  made  in  seconds.  Ship 
commanders  of  today  have  precious  few  seconds  to  identify  an 
approaching  aircraft  and  determine  a  course  of  action. 
Typically  in  military  organizations  these  decisions  are  made 
by  teams . 


Specifically,  in  the  case  of  the  battle  fought  by  naval 
ships  the  team  is  composed  of  the  Composite  Warfare 
Commander (CWC)  and  his  immediate  subordinates:  the  Anti-Air 
Warfare  Commander,  the  Anti-surf ace-Warfare  Commander,  and  the 
Anti-Submarine  Warfare  Commander.  Each  of  the  subordinates 
makes  decisions  and  passes  along  information  to  the  commander. 
He  makes  decisions  about  the  overall  battle  and  passes  down 
information  and  instructions  to  his  subordinates.  Elaborate 
and  integrated  information  systems  have  been  developed  to 
assist  these  commanders  make  their  decisions.  How  commanders 
make  decisions  in  this  computer  assisted  environment  is  a 
growing  concern  in  military  organizations. 

In  order  to  understand  the  dynamics  of  team  decisionmaking 
in  the  Navy  the  Office  of  Naval  Research  has  initiated  the 
Distributed  Tactical  Decision  Making  (DTDM)  program.  (Vaughn, 
1990)  The  purpose  of  this  study  is  to  understand  the  problems 
of  distributed  command  and  control.  The  program  was  initiated 
to  look  at  a  wide  range  of  methodologies .  The  normative  - 
descriptive  method  and  lab  simulation  forms  the  basis  for  the 
current  research. 

A  computer  based  paradigm  designed  to  simulate  "real 
world"  Navy  engagements  has  been  developed  by  a  research  team 
at  the  University  of  Connecticut  and  the  research  corporation 
of  Alphatech.  (Kleinman  and  Song,  1990)  The  Composite 
Warfare  Commander  -  Distributed  Dynamic  Decisionmaking  (CWC- 
DDD)  paradigm  was  the  basis  of  an  experiment  conducted  at  the 


Naval  Postgraduate  School  during  August  1991.  The  Resource 
Allocation  in  Naval  Command  Teams  (RAINCOAT)  experiment  was 
designed  to  study  the  impact  of  decision  making  variables 
using  military  personnel  as  the  decision  makers. 

C.   THE  RAINCOAT  EXPERIMENT 

The  study  of  distributed  decision  making  is  more  then  a 

study  of  the  decision  making  steps  that  individuals  go  through 

to  make  decisions.   In  this  case  the  sum  of  the  parts  can  be 

more  then  the  whole  or  if  things  are  going  wrong  the  sum  can 

definitely  be  less  then  the  whole.    Serfaty  and  Kleinman 

summarized  the  state  of  the  art  in  the  study  of  distributed 

decision  making  at  the  beginning  of  their  study  as  follows: 

The  scientific  study  of  distributed  human  decision 
problems  has  long  been  hindered  by  several  objective 
factors.  The  inherent  complexity  of  the  mathematical 
formulation  and  solution  of  decentralized  problems  in 
control,  detection,  data  fusion,  and  resource  allocation 
theories  has  prevented  the  development  efficient  and 
practical  models  that  could  be  used  as  a  basis  to  predict 
actual  team  performance  in  a  distributed  environment. . . . 
Another  hampering  factor  is  the  lack  of  psychological  or 
cognitive  models  of  team  decisionmaking  behavior.... 
Finally  the  lack  of  comprehensive  empirical  data  has 
prevented  the  development  of  essential  scientific 
hypotheses  related  to  team  decisionmaking  performance. 
(Kleinman  and  Serfaty,  1989) 

The  recognition  of  these  deficiencies  in  this  field  of 

study  and  the  real  world  need  to  better  understand  the 

dynamics  of  team  decisionmaking  lead  to  the  initiation  of  this 

study  by  ONR.   As  stared  earlier,  the  study  concentrates  on 

solving  the  types  of  problems   encountered  in  naval  command 


and  control  systems  which  involve  distributed  decisionmaking. 
The  scenarios  developed  thus  far  use  computer  simulations 
modeling  the  composite  warfare  commander  (CWC)  doctrine 
employed  by  naval  battle  groups.  However,  the  results  of  the 
study  conducted  so  far  have  application  in  many  different 
military  and  civilian  situations  involving  command  and  control 
of  distributed  resources. 

The  RAINCOAT  experiment  sought  to  study  the  dynamics  of 
team  decisionmaking  as  it  was  practiced  by  teams  of  military 
officers .  The  Composite  Warfare  Commander  -  Distributed 
Dynamic  Decisionmaking  (CWC  -  DDD)  paradigm  was  used  to 
present  each  four  person  team  with  an  abstracted  military 
situation  in  which  they  must  respond  to  an  enemy  air  attack. 
The  teams  were  presented  with  unidentified  targets  which  they 
must  identify  as  enemy  or  neutral.  Each  team  received  at 
least  four  hours  of  training  prior  to  recorded  trials  of  the 
experiment .  The  goal  of  this  training  was  to  create  expert 
teams  with  a  complete  understanding  of  the  mechanics  of  the 
simulation.  A  more  complete  description  of  the  experimental 
procedures  used  in  the  experiment  is  contained  in  chapter  III 
of  this  document. 


II.   CURRENT  RESEARCH 

A.   DEVELOPMENT  OF  DDD 

The  current  Composite  Warfare  Commander  -  Distributed 
Dynamic  Decisionmaking  (CWC  -  DDD)  paradigm  is  the  result  of 
five  years  of  study  involving  the  performance  of  model  driven 
basic  experimental  research.  The  development  has  been  a  joint 
effort  between  the  University  of  Connecticut  and  Alphatech 
Inc.  working  under  a  grant  from  the  Office  of  Naval  Research. 
The  paradigm  is  designed  to  support  empirical  research  and 
laboratory  simulation  examining  distributed  decision  making 
issues  in  four-person  hierarchial  teams  of  naval  commanders. 
The  players  plan,  coordinate,  and  allocate  resources  in  a 
simulated  naval  battle  situation. 

The  paradigm  is  implemented  as  a  computer  driven 
interactive  game  in  which  four  team  members  attempt  to  defend 
a  naval  battle  group.  The  paradigm  can  be  manipulated  to 
expose  the  decision  makers  to  different  situations  of 
information  processing  and  resource  allocation  under  different 
organizational  hierarchies  and  information  structures .  The 
decision  makers  are  expected  to  manipulate  limited  shareable 
resources  to  process  a  variety  of  targets  in  a  changing 
environment .   Each   decisionmaker   sits   at   a   different 


workstation  which  displays  the  situation  as  seen  by  each 
decisionmaker.   (Kleinman  and  Song,  1990) 

B.   PAST   EXPERIMENTAL  RESEARCH  IN  TEAM  DECISIONMAKING 

1 .  Experimental     Efforts     in     Hierarchial     Team 
Decisionmaking 

A  compilation  of  the  results  of  current  studies  of 
team  decisionmaking  dynamics  is  contained  in  Table  1 .  This 
table  gives  a  brief  summary  of  the  experimental  research  that 
has  been  conducted  under  the  sponsorship  on  the  Office  of 
Naval  Research. 

2 .  Summary  of  Experimental  Results 

a.  Hierarchical    Information  Processing    (HIP) 

This  experiment  was  an  essential  part  of  a 
coordinated  study  analyzing  team  situation  assessment  and 
information  coordination.  The  experiment  studied  situation 
assessment  in  three-person  teams .  A  team  leader  was  forced  to 
assimilate  information  gained  from  subordinates  with  his  own 
information  and  opinions  to  determine  if  a  contact  was  a 
neutral  or  an  enemy.  The  study  was  designed  to  look  at  the 
effects  of  feedback  (in  the  form  of  subordinates  opinions  and 
confidence  in  their  opinions)  on  the  leaders  assessment  of  the 
situation.  The  experimental  results  supported  the  hypothesis 
that  the  feedback  mechanism  helped  to  insulate  teams  against 
the  lose  of  a  critical  element  of  information.  The  results 
hint  that  with  feedback  team  members  form  a  mental  model  of 


the  other  decision  makers  and  thus  are  able  to  coordinate  team 
actions  with  less  communication.   (Burton  and  Kleinman,  1990) 


TABLE  1 


EXPERIMENTAL  RESULTS  IN  TEAM  DECISIONMAKING 


EXPER 

VARIABLES 

SUBJECTS 

RESULTS 

IMENT 

HIP 

Information 

3  Person 

Feedback  insulates  team 

Feedback 

Teams 

from  information  losses 

HRA 

Tempo 

4  Person 

Leaders  pool  resources 

Teams 

and  positively  affect 
task  division 

INCO 

Information 

4  Person 

Shared  opinions  among 

Teams 

subordinates  reduces 
need  for  leader  and 
increases  effective  use 
of  information 

HITEC 

Time  Stress 

4  Person 

Explicit  coordination 

Leader 

Teams 

is  most  greatly 

Involvement 

affected  by  tempo 

Commo  Structure 

increases 

REST 

Reward  Structure 

2  Person 

Differing  goals  in 

Flat  Team 

teams  forced  an 
increase  in  explicit 
coordination  and 
increased  perceived 
workload. 

CREST 

Performance 

2  Person 

Performance  feedback 

Feedback 

Team 

did  not  greatly  enhance 

Expertise 

team  performance 

Overlap 

ICS 

Information 

4  Person 

Increase  in  external 

structure 

Team 

load  increases  the 

Tempo 

leader  role.  Leader  can 

Command  Strategy 

suffer  information 
overload 

Jb.  Hierarchical    Team  Resource  Allocation    (HRA) 

This  experiment  was  the  concluding  experiment  in  a 
series  of  studies  looking  at  team  resource  allocation,  action 
selection,  and  team  coordination.  This  experiment  examined 
the  role  of  a  leader  in  resource  allocation  problems.  The 
experimental  goal  was  to  determine  if  a  teams  performance 
increased  with  the  introduction  of  a  team  leader  and  a 
hierarchical  structure.  The  leader  became  the  resource 
allocator  with  the  ability  to  force  transfers  of  resources 
among  subordinates .  The  researchers  concluded  that  the 
introduction  of  a  leader  had  a  positive  effect  on  the  teams 
ability  to  coordinate  an  overall  pooling  of  resources .  They 
also  concluded  that  the  team  was  better  able  to  coordinate 
actions  to  process  high  value  tasks.  These  studies  (HIP  and 
HRA)  served  as  the  base  of  information  at  the  beginning  of  the 
research  effort  sponsored  by  the  Office  of  Naval  Research. 
(Miao,  Luh,  and  Kleinman,   1990) 

c.  Information   Coordination   In  Human    Teams    (INCO) 

This  experiment  was  designed  to  study  information 
coordination  in  hierarchical  teams.  The  experiment  used  four- 
person  teams.  The  team  leader  was  asked  to  identify  a  contact 
as  friend  or  foe  based  on  information  passed  to  him  by  his 
subordinates  and  on  his  own  information.  The  subordinates 
were  asked  to  either  identify  the  contact  and  pass  this 
information   to   the   team   leader   or   request   additional 
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information.  This  additional  information  cost  the  team  and 
reduced  their  overall  score.  The  team  goal  was  to  maximize 
that  score.  The  experiment  found  that  when  team  members 
share  their  opinions  the  team  uses  information  more 
effectively  and  the  leader's  involvement  is  much  less 
critical.  The  researchers  also  found  that  when  the 
subordinates  did  not  share  information  the  team  performed 
better  with  active  leader.   (Kleinman  and  others,  19  91) 

d.  Hierarchial    Team  Coordination    (HITEC) 

This  experiment  manipulated  the  time  available  for 
decisions,  the  leader's  role  and  the  communication  structure. 
The  experiment  revealed  that  with  a  35%  increase  in  the  game 
tempo  there  was  a  10%  drop  in  team  performance,  a  25%  drop  in 
the  number  of  communication  messages  and  a  10%  drop  in  the 
database  (task  status)  updates.  The  communication  messages 
with  the  longest  planning  horizon  were  reduced  the  most.  The 
tasks  that  required  more  coordination  were  the  first  to  be 
dropped.   (Wang  and  others,  1991) 

e.  The  Reward  Structure  Experiment     (REST) 

This  experiment  measured  task  processing  in  two 
person  teams  under  an  individual  reward  system  and  under  a 
team  reward  system.  The  different  goals  at  the  local  levels 
forced  more  explicit  coordination,  increased  the  perceived 
workload,  reduced  the  number  of  cooperative  team  actions  and 
lowered  the  overall  performance.   In  other  words  the  teams 
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that   sought   a  team  goal   performed  better   at   both  the 
individual  and  team  levels.   (Kleinman  and  others,  1991) 

f.  Conflict   Resolution   In    Teams    (CREST) 

This  experiment  investigated  conflicts  in  resource 

sharing  in  two  person  teams .   The  team  was  confronted  with  a 

series  of  air  and  subsurface  targets/tasks.  The  team  was 

required  to  identify  and  then  track  each  of  these  tasks  for  a 

specified  period  of  time.  DM1  was  required  to  track  air  tasks 

while  DM2  was  responsible  for  subsurface  tasks.   The  results 

suggested  performance  feedback  does  not  significantly  effect 

team   performance.     Rather   it   reduces   the     degree   of 

cooperative  behavior  in  a  team.  The  experimenters  concluded: 

Feedback  helps  correctly  calibrate  the  mutual  models  that 
decisionmakers  hold  of  each  other  and  that  feedback  is 
filtered  and  interpreted  more  so  at  an  individual  level 
then  at  a  team  level.  (Nodoushani,  Kleinman  and  Serfaty, 
1989) 

g.  Information   and  Command  Strategy    (ICS) 

This  experiment  was  designed  to  investigate  the 
effect  of  different  information  structures  and  command 
strategies  upon  resource  allocation  in  hierarchical  teams . 
Four-person  teams  consisting  of  a  leader  and  three 
subordinates  are  given  limited  resources  to  process  tasks 
within  a  given  area.  The  independent  variables  in  this 
experiment  were  leader  information,  command  options,  and  the 
external  load  applied.  The  experiment  showed  that  if  a  leader 
has  too  much  authority  and  not  enough  information  they  tend  to 
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intrude  into  lower  level  processes .   Additionally  it  showed 
that  too  much  information  can  lead  to  information  overload. 
(Shi,  Luh  and  Kleinman,  1990) 

C.   A  VIEW  OF  TEAM  DECISIONMAKING  IN  COMMAND  AND  CONTROL 
1 .   The  Command  and  Control  Environment 

Tasks  assigned  to  command  and  control  elements  are 
dynamic  in  nature;  they  can  be  diverse  and  completely 
unpredictable.  The  teams  that  are  assembled  to  perform 
command  and  control  are  often  separated  geographically  and 
into  different  functional  responsibilities.  They  make 
decisions  in  an  uncertain  environment  where  there  is  not  a 
easily  defined  correct  answer.  Typically  there  is  an  adversary 
who  will  work  to  confuse  the  decisionmakers  of  the  team.  In 
these  situations  the  team'  s  tasks  can  be  broken  into  two 
distinct  areas .  They  are  situation  assessment  and  resource 
management . 

Situation  assessment  involves  a  continuous  analysis 
and  estimate  of  the  current  situation.  It  requires  that  the 
team  control  information  gathering  resources  and  place  them 
in  positions  that  will  maximize  their  effectiveness.  This 
assessment  requires  that  the  team  have  a  complete 
understanding  of  the  strengths  and  weaknesses  of  the  resources 
available  to  them.  In  a  military  situation  this  involves  the 
use  of  radars,  detection  devices,  IFF  devices  to  identify 
friendly  forces,   special  operation  forces  and  many  other 
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intelligence  gathering  forces  and  equipment.  In  business  this 
can  involve  a  study  of  the  market,  surveys  of  potential 
buyers,  research  of  local  newspapers,  or  even  lobbying  of  the 
government . 

Once  this  information  is  gathered  it  must  be  processed 
to  present  a  clear  picture  of  the  situation.  This  process 
involves  statistical  hypothesis  testing,  pattern  recognition 
and  in  many  cases  judgement  of  the  decisionmakers .  This  step 
can  be  one  of  the  most  difficult  to  implement  because  huge 
amounts  of  data  can  be  collected  in  a  very  short  time.  The 
decisionmaking  team  must  find  a  way  to  extract  the  pertinent 
data  and  assemble  it  into  a  clear  picture  of  the  situation  in 
time  to  affect  a  rapidly  changing  situation. 

Resource  management  is  the  control  and  allocation  of 
an  individual's  resources  in  order  to  utilize  them  in  a 
effective  and  efficient  manner  to  achieve  organizational 
goals.  When  allocating  resources  the  team  must  remember  that 
the  situation  assessment  is  ongoing  and  the  situation  may  be 
changing.  Therefore,  the  teams  must  allow  for  a  strategic 
reserve  of  resources  to  meet  new  demands .  Teams  must  use 
their  judgement,  experience,  and  obtained  data  to  decide  when 
and  where  to  use  this  reserve. 

The  execution  of  the  team  tasks  is  influenced  by  the 
goals  that  the  individual  members  of  the  team  perceive  as 
their  own  and  those  of  the  team  as  a  whole.  The  REST 
experiment  showed  that  those  teams  that  were  able  to  integrate 
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individual  goals  into  the  team  goal  attained  much  higher 
performance  levels.  The  leader  in  this  case  is  challenged  to 
find  common  goals  for  his  subordinates  and  to  define  them  in 
a  way  that  everyone  understands . 

2.   Team  Coordination  -  The  G.R.I.T.  View 

Serfaty  and  Entin  present  a  model  of  team 
decisionmaking  that  defines  a  task  as  a  general  coordination 
problem.  They  define  coordination  as  the  "process  of  managing 
the  overlap  (interdependencies)  among  co-acting 
decisionmakers"  (Kleinman  and  others,  1991) .  Furthermore 
Serfaty  and  Entin  maintain  that  this  coordination  is  an 
essential  element  of  the  team  decisionmaking  process . 

Serfaty  and  Entin  define  four  components  of 
coordination:  goal  coordination  (G)  ,  resource  coordination 
(R)  ,  information  coordination  (I)  and  task/action  coordination 
(T)  .  Goal  coordination  involves  the  team  in  the 
reconciliation  process  between  individual  and  team  goals. 
Resource  coordination  involves  an  accounting  of  available 
resources;  when  and  where  they  will  be  available.  Information 
coordination  involves  the  deciphering  of  incoming  data.  It  can 
also  be  as  simple  as  everyone  agreeing  on  a  common  set  of 
symbols  to  depict  actual  items.  Task/action  coordination 
involves  the  deconf licting  of  action.  For  example,  two  members 
of  an  air  defense  team  should  not  inadvertently  process  the 
same  target  at  the  same  time. 
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The  present  line  of  research  is  investigating  how  this 
coordination  is  facilitated  by  different  environments. 
Specifically,  the  research  is  looking  into  how  coordination  is 
affected  by  the  team's  communication  methods,  the  team's 
information  sharing,  preplanned  rules  or  procedures,  and  the 
team's  hierarchial  structure. 

3 .   Summary  of  Conclusions  from  Current  Research 

a.  Team  Limitations   and  Biases 

Strategies  developed  by  decisionmakers  have  a  short 
planning  horizon;  two  to  three  stages  of  decisionmaking  appear 
to  be  the  limit .  Decision  strategies  developed  by  teams  tend 
to  be  different  then  strategies  developed  by  individuals. 
Individual  decisionmakers  tend  to  believe  that  their  judgement 
of  the  situation  is  the  best  and  that  their  task  is  the  most 
important.  When  possible,  teams  tend  to  overuse  communication 
channels  in  an  attempt  to  reduce  future  uncertainty.  This  is 
particularly  true  when  the  workload  is  light.  Finally,  teams 
tend  to  underestimate  their  ability  to  coordinate  implicitly. 
(Wang,  Serfaty,  Luh  and  Kleinman,   1991) 

Jb.  Team  Adaptation   to    Time   Stress 

The  level  of  stress  that  a  team  encounters 
influences  the  their  performance  in  different  ways.  As  the 
external  load  began  to  increase  the  teams  prioritized  their 
tasks  and  eliminated  those  with  a  low  priority  (Miao,  Luh  and 
Kleinman,  1990) .   At  this  stage  the  teams  seemed  to  prefer 
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processed  information  to  raw  data.  Additionally,  the  amount 
of  communication  among  the  team  members  tended  to  decrease 
sharply.  Decisionmakers  shrank  their  planning  cycle  and 
eliminated  tasks  that  required  extensive  coordination. 
Although  the  teams  were  beginning  to  encounter  problems,  the 
level  of  performance  usually  remained  fairly  high. 

As  the  external  load  increased  even  further  the 
teams  performance  decreased  significantly.  The  teams  tended 
to  disintegrate  into  a  collection  of  individual  decisionmakers 
instead  of  a  cohesive  team.  The  load  was  spread  unequally  in 
order  to  reduce  the  amount  of  coordination  required  between 
members  of  the  team.  This  is  unfortunate  for  the 
decisionmaker  who  happens  to  be  in  the  busy  sector. 
Additionally,  the  error  rate  showed  a  significant  increase  as 
team  performance  decreased.  (Wang  and  others,  1991) 
c.  Communication   Strategy 

As  the  tempo  of  the  external  load  increased  from  a 
low  tempo  to  a  moderate  tempo  the  teams  explicit  coordination 
and  resource  transfer  rates  tended  to  increase.  However,  as 
the  tempo  increased  to  a  high  rate,  the  explicit  coordination 
dropped  to  a  rate  significantly  lower  then  the  low  tempo  rate. 
The  researchers  concluded  that  the  high  tempo  forced  the  team 
members  to  behave  more  like  individual  decisionmakers  then  as 
a  coordinated  team.   (Wang  and  others,  1991) 
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d.  Team  Leader 

The  role  of  the  team  leader  diminishes  as  the 
subordinates  are  provided  with  communication  methods  for  cross 
coordination  or  if  information  sources  are  shared  throughout 
the  team.  The  team  leader  is  affected  by  the  type  and  amount 
of  information  received.  The  team  leaders  authority  must  be 
consistent  with  the  information  that  is  available.  Blanket 
authority  without  information  causes  the  team  leader  to 
intrude  on  the  decisionmaking  of  his/her  subordinates. 
However,  too  much  information  can  quickly  overload  the  team 
leader  and  render  decisionmaking  impossible.  (Shi,  Luh  and 
Kleinman,  1990)  Leaders  at  different  levels  tended  to  hold 
different  perceptions  of  the  team  problem.  These  perceptions 
caused  internal  conflict  in  the  team  and  forced  differing 
coordination  methods  at  the  different  hierarchial  levels  in 
the  team.  (Wang  and  others,  1991) 
4 .   Research  Questions 

The  goal  of  the  RAINCOAT  experiment  is  to  provide 
further  information  into  the  dynamics  of  team  decisionmaking 
as  practiced  by  teams  of  military  decisionmakers.  The  research 
question  is :  what  is  the  impact  of  uncertainty,  differing 
information  flows  and  different  leader  roles  on  team 
performance  and  team  decisionmaking  procedures. 


18 


III.   METHODOLOGY 

A.   EXPERIMENTAL  DESIGN 

The  purpose  of  this  study  was  to  examine  the  effects  of 
certainty  of  target  identif ication,  the  command  structure,  and 
the  role  of  the  leader  on  team  performance,  team  strategy,  and 
team  coordination.  A  double  2x2  repeated-measures  within- 
subject  design  was  used.  As  Figure  1  indicates  the  design 
allows  us  to  examine  three  main  effects  (U,  I,  and  L)  and  two 
interaction  effects  (U  x  I  and  I  x  L) . 

1 .   Independent  Variables 

The  independent  variables  for  this  study  were  (1) 
Neutral/Enemy  Discriminability  Uncertainty  (2)  Team 
Information  Structure  and  (3)  Leader's  Involvement  in  Resource 
Coordination.  Neutral/Enemy  Discriminability  Uncertainty  (U) 
was  established  by  having  all  enemy  aircraft  fly  at  either  0.4 
kft  (low  uncertainty)  or  0 . 5  kft  (high  uncertainty)  .  In  both 
the  low  and  high  uncertainty  environments  neutral  aircraft 
flew  at  0.7  kft.  In  both  cases  sensor  noise  caused  readings 
to  fluctuate  with  a  standard  deviation  of  0.2  kft  and  the 
scale  was  truncated  at  two  standard  deviations.  So,  for 
example,  100%  of  enemy  aircraft  in  a  low  uncertainty  situation 
would  have  readings  from  0.0  to  0.8  while  neutral  aircraft  in 
the  same  situation  would  have  readings  from  0.3  to  1.1.   These 
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Figure  1  -   Experimental  Design 


numbers  show  that  readings  from  0 . 3  to  0 . 8  could  be  either 
enemy  or  neutral  showing  how  uncertainty  is  built  into  the 
study  (see  Figure  2) . 

Team  Information  Structure  (I)  was  termed  either 
Centralized  or  Decentralized.  The  leader's  information  set 
always  included  all  targets  (enemy,  neutral,  and 
unidentified) .  This  independent  variable  pertains  only  to  the 
subordinates'  information  structure.  In  a  Decentralized 
environment  the  subordinates'  information  set  includes  targets 
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Figure  2  -  Neutral /Enemy  Discriminability  Uncertainty 

in  their  own  area  of  responsibility  (including  own  overlap 
areas)  only.  In  a  Centralized  environment  the  subordinates' 
information  set  includes  targets  in  their  own  area  of 
responsibility  and  any  enemy  identified  as  such  in  all  three 
areas . 

Leader  Involvement  in  Resource  Coordination  (L)  was 
termed  either  Active  or  Passive.  In  an  Active  leader  role, 
the  leader  was  involved  in  situation  assessment  and  enemy 
attack  determination,  and  was  allowed  to  take  an  active  part 
in  resource  coordination  and  platform  transfer  among  the 
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subordinates   (including   advice   on   transfers   and   forced 
transfers) . 

In  a  Passive  leader  role  the  leader  was  only  involved 
in  situation  assessment  and  enemy  attack  determination,  but 
was  not  allowed  to  take  part  in  resource  coordination  and 
platform  transfer  among  the  subordinates. 
2 .   Raincoat  Counter-Balanced  Design 

Six  basic  scenarios  were  designed,  with  an  embedded 
two-level  variable:  team  coordination  requirement.  Depending 
on  the  direction  of  the  main  enemy  attack,  either  resource  or 
task  coordination  is  being  stressed.  The  scenarios  were 
randomized  across  teams  and  experimental  conditions  in  such  a 
way  that  every  Exam  was  treated  once  by  each  of  the  six 
experimental  conditions  and  by  each  scenario  once.  Three  of 
these  scenarios  stressed  resource  coordination  by  placing  the 
primary  attack  in  the  middle  of  a  DM' s  own  area  of 
responsibility  (each  DM  was  given  one  primary  attack) . 
Resource  coordination  requires  the  DM  to  basically  work  on 
his/her  own  and  request  additional  resources  as  required.  The 
other  three  scenarios  stressed  task/action  coordination  by 
placing  the  primary  attack  in  an  overlapping  sector  (each 
overlap  sector  was  given  one  primary  attack) .  Task/action 
coordination  requires  two  adjacent  DM' s  to  work  together  in 
deciding  who  is  going  to  work  on  which  target (s) .  Table  2 
shows  how  the  design  was  counter-balanced  for  this  experiment. 
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TABLE  2 


COUNTER-BALANCED  DESIGN 


SCENARIO 

1 

2 

3 

4 

5 

6 

Experimental 

Condition 

(U,  I,  L) 

11 

12 

13 

21 

22 

23 

111 

A 

B/G 

D 

C 

E 

F 

121 

F 

D 

C 

E 

A 

B/G 

211 

E 

F 

B/G 

D 

C 

A 

221 

D 

E 

A 

F 

B/G 

C 

212 

C 

A 

F 

B/G 

D 

E 

222 

B/G 

C 

E 

A 

F 

D 

B .   SUBJECTS 

Subjects  for  the  experiment  consisted  of  27  active  duty 
military  officers  and  one  civilian  employee  of  the  National 
Security  Agency   enrolled   in   the   Command,   Control,   and 
Communications   (C3  curriculum  at  the  Naval   Postgraduate 
School,  Monterey  California.  Five  of  the  subjects  were  female 
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and  23  male.  The  subjects  were  allowed  to  freely  divide  into 
seven  teams  of  four  members  each.  Within  each  group,  one 
person  was  picked  to  be  the  leader  (DM0)  and  the  other  members 
played  the  roles  of  three  subordinate  decisionmakers  (DM1, 
DM2,  and  DM3) .  During  the  training  and  practice  sessions  the 
team  members  were  allowed  to  freely  change  positions  to  get  a 
feel  for  what  was  required  of  each  position.  Once  the  real 
experimental  sessions  began  changing  of  positions  was 
prohibited.  The  level  of  experience  was  controlled  by  using 
the  Composite  Warfare  Commander  -  Distributed  Dynamic 
Decisionmaking  (CWC-DDD)  experimental  paradigm.  None  of  the 
subjects  had  previously  used  the  CWC-DDD  simulation. 

C .   APPARATUS 

In  our  experiment  the  CWC-DDD  simulation  software  was  set- 
up to  run  on  a  Sun  Microsystems  SPARC  1+  server  with  four 
SPARCstation  SLC  diskless  workstations,  each  having  a 
monochrome  monitor  and  an  optical  mouse,  all  linked  together 
through  an  ethernet .  The  software  was  written  to  run  under 
SunOS  and  the  Sunview  operating  environment.  Team  members 
participated  in  the  simulation  using  the  workstations  while 
the  server  ran  the  simulation.  The  network  and  experiment 
were  in  a  room  where  noise  was  kept  to  a  minimum  and 
distractions  that  might  influence  the  outcomes  were  minimized. 
Each  DM  had  adequate  room  and  was  kept  from  seeing  the  other 
DM' s  monitors. 
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D .   TASKS 

The  tasks  carried  out  by  the  subjects  were  conducted  using 
the  CWC-DDD  experimental  paradigm.  The  CWC-DDD  simulation 
creates  a  tactical  environment  that  replicates,  with  a 
considerable  level  of  abstraction,  decisionmaking  problems 
encountered  by  commanders  in  naval  battle  groups .  It  was 
developed  by  a  research  team  at  the  University  of  Connecticut 
for  use  in  the  CREST  and  ICS  experiments  to  provide  a  rather 
abstracted  military  environment .  Certain  features  of  the  CWC- 
DDD  simulation  were  greatly  simplified  or  even  removed  to 
allow  us  to  focus  on  a  particular  aspect  of  the  team 
decisionmaking  process  for  purposes  of  the  RAINCOAT  (Resource 
Allocation  In  Naval  Command  Teams)  experiment. 

1 .   Problem  Domain 

The  problem  domain  constructed  for  the  RAINCOAT 
experiment  consisted  of  three  parts: 


•  Anti-Air  Problem  (Inner/Outer  Air  Battle) .  The  objective 
of  the  blue  team  is  to  defend  a  defined  area  around  an 
aircraft  carrier  against  an  orange  air  threat . 

•  Four-Person  Blue  Teams.  A  leader  (Anti-Air  Warfare 
Commander  -  DM0)  and  three  subordinates  (DM1,  DM2,  and 
DM3)  responsible  for  three  geographical  sectors  (see 
Figure  3)  . 

•  Orange  Attack  Scenario  is  pre-programmed  in  the  simulation 

(there  were  six  pre-programmed  scenarios  used  for  this 
experiment) .  Targets  are  maneuvering  in  the  theater  of 
operations.  However,  the  targets  are  not  reactive  to  the 
actions  of  the  blue  team. 
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2 .   Phases 

Each  experimental  session  is  broken  down  into  three 
phases:  Planning  (PL),  Situation  Assessment  (SA) ,  and  Threat 
Prosecution  (TP)  . 

a.  Planning    (PL)    Phase 

During  the  PL  phase  the  blue  team  receives  a 
detailed  description  of  their  own  resource  capabilities  (how 
many  and  what  type  of  platforms  and  subplatf orms)  ,  their 
relative  emplacement  in  future  operations,  a  brief 
intelligence  assessment  of  the  overall  situation  (whether  the 
enemy  is  expected  to  attack)  ,  and  a  general  description  of  the 
probable  enemy  attack  tactics  (that  the  enemy  will  use  a 
primary  and  diversionary  attack  path) .  The  job  of  the  blue 
team  is  to  prepare  a  coordinated  defense  plan  based  on  this 
preliminary  information,  answer  a  few  questionnaires, 
preposition  their  platforms,  and  wait  for  the  start  of  the  SA 
phase . 

Jb.  Situation  Assessment    (SA)    Phase 

During  the  SA  phase  the  job  of  the  subordinates  is 
to  allocate  their  sensors  (platforms  and  subplatf orms)  and  to 
take  sensor  readings  in  order  to  make  a  neutral  or  enemy 
assessment  on  a  target-by-target  basis,  in  their  sector  of 
responsibility.  Figure  3  presents  an  example  situational 
display  for  a  subordinate  commander  showing  the  division  of 
responsibility  using  pie-shaped  geographical  sectors  for  a 
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typical  CWC— DDD  simulation.  In  the  next  step  the  subordinates 
report  their  assessment  to  the  team  leader  (interpret  their 
sensor  readings  as  either  enemy  or  neutral,  what  size  or 
damage-inducing  potential  the  target  has,  and  how  confident 
they  are  in  their  decision)  .  The  job  of  the  leader  is  to 
request  more  information  on  a  particular  target  if  necessary 
and  once  satisfied  with  the  accuracy  of  all  subordinates' 
reports,  to  look  at  the  global  situation  and  make  an  attack 
determination.  He  should  decide  if  the  blue  forces  are  under 
attack,  where  is  the  main  avenue  of  attack  or  threat  axis  (in 
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what  sector) ,  where  is  the  eventual  secondary  or  diversionary 
attack,  and  when  the  blue  team  should  start  threat  prosecution 
for  the  purpose  of  self-defense. 

c.    Throat   Prosecution    (TP)    Phase 

During  the  TP  phase  the  job  of  the  subordinates  is 
to  allocate  their  subplatforms  to  engage  the  threat  as  defined 
by  the  leader.  Each  subplatform  has  a  radius  of  lethality 
where  an  attack  can  be  undertaken  with  a  certain  probability 
of  kill.  To  do  their  job  most  effectively  they  must  transfer 
resources  (subplatforms)  to  the  heaviest  attack  area  and  under 
the  responsibility  of  that  area  commander.  They  may  have  to 
cooperate  on  some  defense  missions  by  pooling  their  resources 
together  against  a  certain  threat  type.  The  job  of  the  team 
leader  is  to  ensure  that  the  main  goal  of  defending  the 
carrier  is  achieved.  To  do  so,  he  may  have  to  enforce 
resource  transfer  among  subordinates,  set  priorities  on 
threats,  and  coordinate  the  subordinate's  actions.  A  certain 
number  of  strength  points  are  taken  away  from  the  team  when  a 
threat  penetrates  the  inner  defense  zone,  the  blue  team 
attacks  a  neutral,  or  the  blue  team  does  not  allocate  the 
proper  amount  of  resources  when  attacking  an  enemy  target . 
The  objective  of  the  team  is  to  maximize  the  number  of 
strength  points  remaining  at  the  end  of  a  scenario.  At  the 
completion  of  the  trial  a  subjective  workload  assessment  is 
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completed  by  each  member  of  the  team  and  finally  a  post- 
engagement  questionnaire  is  distributed  to  team  members. 

E.   EXPERIMENTAL  PROCEDURE 

Initial  exposure  to  the  experiment  was  accomplished 
through  a  briefing  which  covered  very  general  characteristics 
of  the  simulation.  Scheduling  and  other  coordination  was 
accomplished  during  this  session  so  that  each  team  would  be 
able  to  provide  a  minimum  of  two  hours  per  week  for  five 
consecutive  weeks  to  the  experiment .  Each  team  was  required 
to  provide  a  minimum  of  ten  hours  to  the  experiment,  four 
hours  for  training/practice  and  six  hours  for  actual  data 
collection . 

1 .   Training  and  Practice  Sessions 

The  first  part  of  team  training  involved  a  walk 
through  using  hard  copy  screen  dumps  from  an  actual  session. 
This  method  was  used  to  explain  each  component  of  the  screen 
to  all  members  simultaneously.  Different  screen  dumps  showed 
various  stages  of  the  scenario  and  each  window  that  a  DM  might 
see  was  fully  explained.  This  portion  of  the  first  session 
lasted  from  45-60  minutes.  After  completion  of  the  walk 
through,  each  team  member  was  seated  at  a  station  and  the 
simulation  was  started  (using  training  scenarios)  .  During 
this  first  (and  all  subsequent  practice  sessions)  run  of  the 
simulation  two  tutors  were  present  to  show  different 
techniques  and  answer  any  questions  posed  by  the  subjects. 
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This  part  of  the  session  lasted  35  minutes.  Finally,  to 
complete  the  first  two  hours  of  training,  a  practice  session 
was  started  and  the  team  was  allowed  to  play  freely,  asking 
questions  as  necessary  and  having  particular  points  stressed 
by  the  tutors.  During  the  training/practice  sessions  team 
members  were  allowed  to  freely  discuss  anything  about  the 
simulation.  The  second  training/practice  session  normally 
started  with  a  review  of  the  previous  session.  The  remainder 
of  the  session  consisted  of  running  consecutive  trials  until 
the  two  hours  was  completed.  By  this  point,  most  teams  were 
confident  in  their  ability  to  continue  on  with  the  data 
collection  sessions.  However,  one  team  asked  for  another 
training  session  to  work  on  some  areas  with  which  they  were 
having  trouble . 

2 .   Data  Collection  Sessions 

These  sessions  started  out  with  briefings  on  the 
various  parameters  of  the  current  session  and  pre-engagement 
questionnaires.  During  these  trials  no  talking  was  allowed 
between  DM' s  except  through  communications  provided  in  the 
simulation.  Each  trial  lasted  approximately  35  minutes; 
however,  60  minutes  were  allotted  for  pre-  and  post-engagement 
questionnaires,  a  short  break  between  trials,  and  any  strategy 
discussions  that  the  team  conducted  during  the  planning  phase. 
Data  was  collected  by  the  network  and  stored  for  later 
analysis . 
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3.   Post  Data  Collection 

After  completion  of  all  data  collection  sessions  a 
short  demographic  questionnaire  was  completed  by  each  subject 
and  team  strategies  were  discussed. 

F.   DEPENDENT  VARIABLES 

The  dependent  variables  for  this  experiment  were  collected 
in  order  to  capture  the  team  behavior  during  each  experimental 
scenario.  They  can  be  divided  into  three  categories: 
Performance,  Strategy,  and  Coordination. 

1 .   Team  Performance  Measures 


Final  Team  Strength  [%] .   This  is  the  most  global  measure 
of  team  performance.    It  is  the  one  that  each  team 
attempts  to  maximize.    At  the  start  of  the  scenario 
(time=0) ,   the  team  is  given  an  initial  strength  S0f 
representing   some   aggregate   value   of   the   defensive 
strength  of  the  battle  group.   As  the  battle  progresses, 
the  team  strength  is  being  reduced  by  the  damage  caused  by 
enemy  forces  as  well  as  by  mistakes  made  by  the  blue  team, 
such  as  attacking  a  neutral  task.    The  relative  team 
strength  at  any  time  is  computed  as: 

S0-AS 


S0 


where  AS  is  the  loss  at  time  t .   This  loss  is  a  function 
of  the  number  of  threats  that  penetrated  the  inner 
defense,  the  number  of  neutral  tasks  attacked  by  the  blue 
force,  and  the  number  of  enemy  targets  that  were  attacked 
with  inadequate  resources  by  blue  forces . 

•  Number  of  False  Alarms  []  .  This  reflects  a  Type  II 
misclassif ication  error  (target  identified  as  enemy  when 
it  is  really  neutral) . 
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•  Number  of  Correct  Enemy  Identifications  [] 


•  Number  of  Misdetections   []  .    This  reflects  a  Type  I 
misclassif ication  error  (target  identified  as  neutral 
when  it  is  really  enemy) . 


•  Number  of  Correct  Neutral  Identifications  []. 


•  Correct  Enemy  Classification  Ratio  [%] .   This  measure  is 

computed  as  the  ratio  of  correctly  classified  enemies 

(Low,  Medium,  and  High  damage-producing  capability)  to  the 

total  number  of  enemy  targets  that  are  identified  as  such. 


•  Number  of  Targets  Unidentified  [] 


•  Correct  Primary  Attack  Determination  [0/1] .  If  the  leader 
correctly  identifies  the  axis  of  the  primary  attack  P  (  +  /- 
1  sector) ,  then  this  variable  takes  a  value  of  1, 
otherwise  it  is  0. 


•  Correct  Diversionary  Attack  Determination  [0/1] .  If  the 
leader  correctly  identifies  the  axis  of  the  diversionary 
attack  D  (+/-  1  sector) ,  then  this  variable  takes  a  value 
of  1,  otherwise  it  is  0. 


Primary/Diversionary  Attack  Confusion  [0/1].  If  the 
leader  incorrectly  determines  the  axis  of  the  primary 
attack  p  on  the  true  diversionary  attack  D  axis  (  +  /-  1 
sector) ,  then  this  variable  takes  a  value  of  1,  otherwise 
it  is  0 . 


•  Diversionary/Primary  Attack  Confusion   [0/1] .     If  the 
leader  incorrectly  determines  the  axis  of  the  diversionary 
attack  D  on  the  true  primary  attack  axis  P  (+/-  1  sector)  , 
then  this  variable  takes  a  value  of  1,  otherwise  it  is  0. 


•  Average  Attack  Score  on  Enemy  Targets  [points] .  This 
score  is  the  function  of  the  value  of  the  enemy  target 
(damage-inducing  potential)  and  the  amount  of  attack 
resources  used  on  the  target.  For  enemy  target  k,  the 
score  is  computed  as 


32 


SCK-VALK*REFKf 

where  VAL  is  the  target  value  (Low=l-4,  Medium=4-6,  and 
High=6-8) ,  and  REF  is  the  resource  effectiveness  for 
target  k .  REF  is  defined  in  Average  Resource 
Effectiveness . 


•  Total  Number  of  Attacks  of  Enemy  Targets  [] . 


•  Number  of  Attacks  of  Enemy  Targets  in  the  Primary  (P) 
Attack  Axis  [] . 


•  Number  of  Attacks  of  Enemy  Targets  in  the  Diversionary  (D) 
Attack  Axis  [ ] . 


•  Total  Leakage  [%]  .    Number  of  true  Enemy  targets  that 
penetrated  the  Inner  Defense  Zone/Total  number  of  threats. 


•  Total  Leakage  in  (P)  [%] . 

•  Total  Leakage  in  (D)  [%] . 

•  Total  Number  of  Attacks  of  Decoys  [] . 

•  Total  Number  of  Attacks  of  Neutrals  [] . 

•  Number  of  Attacks  on  Low  Threat  Targets  [] . 

•  Number  of  Attacks  on  Medium  Threat  Targets  [] . 

•  Number  of  Attacks  on  High  Threat  Targets  [] . 
2 .   Team  Strategy  Measures 

•  Average  Resource  Effectiveness  (AREF)  [%]  .  An  enemy 
target  requires  the  allocation  of  a  pre-determined  amount 
of  resources  (weapons)  to  be  attacked  with  maximum 
effectiveness  (i.e.,  completely  destroyed)  .  The  amount  of 
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required  resources  is  a  function  of  the  damage— producing 
capability  of  the  target.  This  capability  is  indicated  by 
the  second  attribute  of  the  target,  i.e.,  its  value 
(Low=2-4,  Medium=4-6,  and  High=6-8) .  Taking  into  account 
that  each  X  subplatform  contains  1  resource,  and  each  Y 
subplatform  contains  2  resources,  Table  3  describes  the 
resource  requirements  for  attacks. 


TABLE  3 


RESOURCE  REQUIREMENTS 


ENEMY  TARGET 
CLASS 

LOW  (2-4) 

MEDIUM  (4-6) 

HIGH  (6-8) 

RESOURCES 
REQUIRED 

1 

2 

3 

SUBPLATFORMS 
NEEDED 

IX  or  1Y 

2X  or  1Y 

3X  or  1X+1Y 

The  resource  effectiveness  of  an  attack  on  an  enemy  target 
is  computed  as  the  square  of  the  ratio  of  resources 
allocated/resources  required.  For  example,  suppose  that 
a  high-valued  enemy  has  been  identified.  Three  resources 
are  required  for  a  complete  kill.  If  a  DM  allocates  IX  + 
1Y  platforms  (three  resources) ,  the  resource  effectiveness 
(REF)  is  (3/3)2=100%.  If  a  DM  allocates  only  1Y  platform 
(two  resources),  the  REF  is  (2/3)2=44%.  There  is  no  bonus 
or  penalty  for  an  over-allocation  of  resources.  For 
instance  if,  in  the  case  described  above,  the  DM  allocates 
4X  platforms  when  only  three  are  required,  the  REF  will 
still  be  100%.  The  AREF  is  computed  as  the  average  of  all 
REF  on  attacks  on  enemy   aircraft  by  the  blue  team. 

AREF  on  Low  Threat  targets  [%] . 


AREF  on  Medium  Threat  Targets  [%] 
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•  AREF  on  High  Threat  Targets  [%] . 

•  Number  of  Attacks  by  Platforms  [] 


•  Average  Number  of  Platforms  used  for  Identification  [] 
Averaged  across  all  targets  identified. 


•  Average  Team  Latency  [sec] .  Delay  between  the  appearance 
of  a  target  and  the  very  first  team  action  on  that  target 
(communication,  identif ication,  or  attack) .  When  averaged 
across  all  targets,  it  represents  a  measure  of  the  overall 
reaction  time  of  the  team. 


•  Average  Identification  Delay  [sec] .  Delay  between  the 
appearance  of  a  target  and  the  first  team  identification 
of  that  target.  Averaged  across  all  tasks  identified  by 
the  team. 


•  Time  of  First  Primary  (P)  Attack  Determination  [sec] 


•  Time   of   First   Diversionary   (D)   Attack   Determination 
[sec]  . 


•  Number  of  Attack  Determinations  [] . 

•  Ratio  of  Targets  Identified  as  Enemy  but  not  Attacked  [%] 

•  Number  of  Direct  Attacks  by  Team  Leader  [] . 


•  Average  Attack  Location  [nm] .  Distance  between  an  attack 
location  and  the  center  of  the  battle  group  (A/C) . 
Averaged  across  all  tasks  attacked  by  the  team. 


•  Average  Identification  Confidence  [] 


•  Average   Number   of   Identification   Actions   per   Task 
Identified  [ ] . 
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3 .   Team  Coordination  Measures 

Total  Communication  Rate  [msg/min] .  Total  team 
communication  traffic  average  over  the  length  of  the 
scenario.  (Note:  each  broadcast  counts  as  three 
messages)  . 


•  Number  of  Messages  per  Target  Attacked  [] 

•  Number  of  Platform  Requests  []  . 


•  Number  of  Advices  []  .  Number  of  "advise  transfer" 
messages  issued  by  the  leader  to  transfer  a  platform 
between  subordinates . 


•  Vertical  Downward  Communication  [].   Number  of  messages 
issued  by  the  leader  to  the  subordinates. 


•  Vertical  Upward  Communications  [].    Number  of  messages 
issued  by  the  subordinates  to  the  leader. 


•  Horizontal  Communication  []  .  Number  of  messages  issued  by 
a  subordinate  to  another  subordinate. 


•  Number  of  Platform  Transfers  by  Subordinates  [] . 

•  Number  of  Platform  Transfers  by  the  Leader  [] . 


•  Number  of  Wasted  Attacks  []  .  A  wasted  attack  may  occur  in 
the  overlap  sectors  if  a  DM  attempts  to  attack  a  target 
already  under  attack  by  another  DM.  This  measure  is 
indicative  of  a  lack  of  planning  and  attack  coordination 
among  the  team  members . 
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IV.   RESULTS 

A.   DATA  ANALYSIS 

Due  to  the  large  quantity  of  dependent  variables  in  the 
data  set  it  was  necessary  to  reduce  the  number  to  a  more 
manageable  size.  The  experiment  began  with  4  9  dependent 
variables  and  through  heuristic  methods  was  reduced  to  15. 
The  original  data  set  contained  several  dependent  variables 
that  were  subsets  of  other  dependent  variables.  These  subsets 
provided  redundant  information  that  was  of  limited  value  and 
were  eliminated  from  further  consideration.  The  values  of 
several  other  dependent  variables  were  discovered  to  depend 
upon  the  experimental  design  and  were  also  eliminated.  An 
examination  of  the  means  of  the  remaining  variables  showed 
that  the  values  of  several  of  the  variables  changed  little 
regardless  of  the  independent  conditions  and  were  removed  from 
consideration.  Two  variables  were  not  considered  because 
their  values  were  affected  by  mechanical  or  software  error. 

Table  4  shows  the  dependent  variables  that  remained,  what 
they  represented,  their  means,  and  their  standard  deviations. 
The  data  that  remained  were  analyzed  using  an  univariate 
analysis  of  variance  (ANOVA)  computed  using  the  General  Linear 
Models  Procedure  of  the  SAS  statistical  software  program  (SAS 
Institute  Incorporated,  1988) . 
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TABLE  4 


FINAL  DEPENDENT  VARIABLE  LIST 


NUMBER 

DEFINITION 

MEAN 

STDEV 

1 

Final  Team  Strength 

64.67 

21.84 

2 

Number  of  False  Alarms 

5.12 

3.30 

5 

Number  of  Correct  Neutral  Identifications 

19.71 

3.27 

6 

Correct  Enemy  Classification  Ratio 

0.80 

0.16 

24 

Average  Resource  Effectiveness 

0.88 

0.10 

30 

Average  Team  Latency  [sec] 

278.84 

81.72 

32 

Time  of  First  P  Attack  Determination  [sec] 

962.07 

222.42 

38 

Average  ID  Confidence 

1.92 

0.26 

39 

Average  Number  of  ID  Actions  per  Task 
Identified 

2.68 

0.82 

40 

Total  Communication  Rate 

0.  67 

0.48 

41 

Number  of  Messages  per  Target  Attacked 

1.69 

1.36 

44 

Vertical  Downward  Communication 

9.12 

12.60 

45 

Vertical  Upward  Communications 

4.05 

4.19 

46 

Horizontal  Communications 

10.24 

6.98 

47 

Number  of  Platform  Transfers  by  Subordinates 

1.95 

1.85 

The  ANOVA  procedure  produces  a  value,  p,  which  represents 
the  probability  of  making  an  error  in  claiming  that  a 
dependent  variable  is  affected  by  different  levels  of  an 
independent  variable.  A  value  of  p  <  0.05  is  considered 
significant.  A  value  of  0.05  <  p  <  0.1  is  considered 
marginally  significant.  The  ANOVA  method  also  identifies 
significant  interactions  between  two  or  more  independent 
variables.   (Miao,  Luh  and  Kleinman,  1990) 
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B.   EXPERIMENTAL  RESULTS 

Two  different  approaches  were  taken  in  conducting  the 
ANOVA.  The  first  approach  considers  between  cell  interactions 

(see  figure  1) ,  while  the  second  approach  involves  between 
team  comparisons.  A  third  approach  was  conducted  using 
correlation  techniques  and  an  analysis  of  the  processes  which 
affected  team  performance . 

1 .   Between  Cells  Analysis 
a.  Summary  Statistics 

Summary  statistics  (means  and  standard  deviations) 
for  each  of  the  15  dependent  variables  are  given  in  table  5. 
The  highest  final  team  strength  were  attained  in  situations 
with  an  active  leader  and  low  uncertainty.  In  situations  with 
an  active  leader  and  high  uncertainty  there  was  a  significant 
difference  in  final  team  strength  between  a  decentralized 
information  structure  and  a  centralized  information  structure 

(54.00  vs.  63.29)  (See  figure  4).  Teams  in  the  former 
situation  (high  uncertainty,  decentralized  information 
structure,  and  active  leader  cell)  also  showed  considerably 
less  team  latency  when  the  information  structure  was 
decentralized  as  opposed  to  all  other  cells.  Under  conditions 
of  high  uncertainty  the  first  primary  attack  determination 
took  longer  when  the  leader  was  passive  and  the  information 
structure  was  decentralized  (1063.43  sec)  or  if  the  leader 
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TABLE  5 
SUMMARY  STATISTICS  -  BETWEEN  CELLS  ANALYSIS 

MEANS 
STANDARD  DEVIATIONS 


CELLS 

LOW 

OECEHT 
ACTIVE 

LOU 

CENT 
ACTIVE 

PASS 
DECENT 
■  IGB 

PASS 
CENT 
HOB 

■  IGB 

DECENT 

ACTIVE 

BIGB 

CENT 
ACTIVE 

1 

73.00 
24.30 

70.71 
25.36 

61.16 

21.90 

65.14 

20.92 

34.00 
13.47 

63.29 
25.41 

2 

2.29 

1.B9 

3.14 

1.3S 

5.16 

2.67 

6.29 
3.59 

7.14 

4.60 

6.00 
2.45 

5 

22.57 
1  .90 

21.71 
1.25 

19.14 

2.67 

14.57 
3.46 

17.16 
4.60 

11.13 

2.37 

6 

0.14 

0.17 

0.10 
0.15 

0.61 

0.14 

0.77 
0.  16 

0.77 
0.21 

0.14 
0.10 

24 

0.19 

0.09 

0.49 
0.04 

0.15 

0.13 

0.91 

0.09 

0.67 
0.01 

0.13 
0.11 

30 

269.70 
95.96 

299.20 
79.32 

272.69 
46.23 

276.01 
43.95 

2SS.1I 
7  4.54 

217.91 
95.05 

32 

964.71 
133.23 

1001.43 

209.14 

1063.43 

261.91 

•41.14 

211. 95 

117.29 
229.21 

1003.43 

115.57 

36 

2.15 

0.22 

2.13 
0.17 

1.12 

0.26 

1.79 
0.23 

1.77 
0.23 

1.64 

0.16 

39 

2.55 

0.  41 

2.54 
0.  65 

2.49 

0.47 

2.72 

0.93 

2.57 

0.92 

2.76 

0.15 

40 

0.71 
0.  31 

0.  66 

0.  63 

0.57 
0.52 

0.42 

0.24 

0.62 

0.65 

0.13 

0.37 

41 

1.61 
0.75 

2.26 

2.06 

1.93 

2.07 

0.93 

0.57 

1.40 
1.15 

1.95 

0.71 

44 

11.14 

9.60 

11.57 
14.23 

7.43 
15.69 

3.00 
4.36 

10.29 

17.75 

11.29 

11.69 

45 

3.57 
2.42 

5.71 
5.46 

2.16 

3.  34 

3.14 

2.91 

3.16 

4.67 

5.14 
5.34 

46 

10.14 
6.74 

12.66 
10.35 

9.57 

5.  74 

1.57 
2.70 

7.57 
5,  44 

12.71 
9.14 

was  active  and  the  information  structure  was  centralized 
(1003.43  sec)  as  opposed  to  a  passive  leader  with  a 
centralized  information  structure  (848.14  sec)  or  an  active 
leader  with  decentralized  information  structure  (887.29 
sec) (See  figure  5) .  Under  high  uncertainty,  with  an  active 
leader  more  identification  actions  were  taken  per  task  with 
centralized  information  than  with  decentralized  information; 
however,  with  a  passive  leader  more  identification  actions 
were  taken  per  task  with  decentralized  information  than  with 
centralized  information  (See  figure  6) .  With  a  passive  leader 
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Figure  4  -  Final  Team  Strength 

and  high  uncertainty  there  were  significantly  more  vertical 
communications  in  decentralized  (7.43)  as  opposed  to 
centralized  (3.00)  information  structures.  With  an  active 
leader  there  were  more  horizontal  communications  with 
centralized  information  than  with  decentralized  information; 
however,  with  a  passive  leader  there  were  more  horizontal 
communications  with  decentralized  information  than  with 
centralized  information  (See  figure  7) . 
Jb.  Analysis   of  Variance 

Table  6  summarizes  the  results  of  the  univariate 
analysis  of  variance  (ANOVA)  for  the  selected  dependent 
variables .    There  was  a  difference  in  the  number  of  false 
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Figure  5  —  Time  of  First  Primary  Attack  Determination 


alarms  found  in  each  of  the  cells  at  the  0 . 0234  significance 
level .  This  shows  that  in  low  uncertainty  situations  the 
number  of  false  alarms  is  markedly  less  than  in  high 
uncertainty  situations .  Supporting  the  same  hypothesis  is  the 
fact  that  the  number  of  correct  neutral  identifications  was 
significant  at  0.0196  significance  level.  This  shows  that  in 
low  uncertainty  situations  teams  were  better  able  to  identify 
neutral  tasks  than  in  high  uncertainty  situations  regardless 
of  the  information  structure  or  leader  role.  Average 
identification  confidence  was  found  to  be  significant  at  the 
0.0020  level.  This  shows  that  in  low  uncertainty  situations 
teams  had  higher  average  identification  confidence  than  in 
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Figure  6  -  Average  ID  Actions  per  Task  Identified 

high  uncertainty  situations.  The  dependent  variables  which 
showed  significance  in  the  between  cells  analysis  all  measured 
an  impact  of  uncertainty.  Dependent  variables  measuring 
impacts  of  the  other  independent  variables  did  not  show 
significance  as  measured  by  the  ANOVA' s .  This  supports  the 
hypothesis  that  only  uncertainty  has  a  significantly  different 
impact  between  the  cells . 

c.  Team  Composition   Summary 

Table   7   shows   the   composition   of   each  team 
participating  in  the  experiment.   This  table  shows  time-in- 
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Figure  7  -  Horizontal  Communication 

service,  branch  of  service,  specialty,  team  average  time-in- 
service,  tactical  rating,  and  strategy  rating.  Tactical 
rating  is  a  percentage  of  the  team  members  that  are  trained  to 
serve  in  tactical  positions .  This  includes  Surface  Warfare 
Officer  (SWO) ,  Naval  Flight  Officer  (NFO)  ,  Air  Defense 
Artillery  Officer  (ADA) ,  Pilot/Electronic  Warfare  Officer 
(EW)  ,  Anti-Air  Warfare  Officer  (AAW) ,  and  Infantry  Officer 
(INF) .  The  non-tactical  positions  included  Computer  Related 
Specialty  Officer  (COMP) ,  Communications  Officer  (COMMS) , 
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TABLE  6 


ANOVA  -  BETWEEN  CELLS  ANALYSIS 


DEPENDENT 
VARIABLE 

SUM  OF  SQUARES 

F  VALUE 

Pr  >  F 

1 

1608.76 

0.65 

0.6668 

2 

130.97 

2.99 

0.0234 

5 

132.28 

3.11 

0.0196 

6 

0.03 

0.26 

0. 9309 

24 

0.02 

0.42 

0.8332 

30 

6333.03 

0.17 

0.9719 

32 

225045.93 

0.90 

0.4926 

38 

1.09 

4.73 

0.0020 

39 

0.63 

0.17 

0.9733 

40 

0.98 

0.83 

0.5382 

41 

7.  98 

0.84 

0.5271 

44 

395.26 

0.47 

0.7990 

45 

45.33 

0.48 

0.7860 

46 

163.33 

0.64 

0.6703 

Instructor  (INST) ,  and  Public  Affairs  Officer  (PAO) .  The 
strategy  rating  information  was  gathered  through  team 
debrief ings  following  their  last  experimental  trial .  Each 
team  was  asked  to  describe  their  strategy  for  handling  overlap 
areas,  transferring  resources,  and  assigning  confidence  levels 
to  identified  tasks.  Some  of  the  teams  had  a  coherent 
strategy  while  others  did  not. 
d.  Summary  Statistics 

Summary  statistics  (means  and  standard  deviations) 
for  each  of  the  15  dependent  variables  are  given  in  table  8 . 
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TABLE  7 
TEAM  COMPOSITION 


TEAM 

DM  # 

TIME 

BRANCH 

PRIMARY 
TRAINING 

AVG  TIME 

TACTICAL 

STRATEG 
Y 

A 

0 

6.5 

USN 

SWO 

10.6 

50% 

YES 

1 

6 

USN 

SWO 

2 

13 

USAF 

COMP 

3 

17 

USAF 

COMP 

B 

0 

8 

USAF 

COMMS 

10.3 

50% 

NO 

1 

12 

USN 

NFO 

2 

5 

USN 

SWO 

3 

16 

USAF 

COMMS 

C 

0 

5.75 

USMC 

ADA 

6.1 

75% 

YES 

1 

6 

USN 

EW 

2 

5 

USAF 

COMP 

3 

7.5 

USN 

AAW 

D 

0 

7 

USN 

INST 

9.0 

0% 

NO 

1 

16 

USAF 

COMMS 

2 

7 

USAF 

COMMS 

3 

6 

USAF 

COMMS 

E 

0 

11 

USN 

NFO/AAW 

10.4 

100% 

YES 

1 

16 

USMC 

INF 

2 

6.5 

USN 

SWO 

3 

8 

USA 

INF 

F 

0 

6 

USN 

SWO 

6.75 

75% 

YES 

1 

7 

USAF 

COMMS 

2 

6 

USN 

SWO 

3 

8 

USN 

SWO 

G 

0 

6 

NSA 

COMMS 

6.1 

50% 

NO 

1 

4 

USN 

SWO 

2 

5.5 

USN 

SWO 

3 

9 

USN 

PAO 

The  three  teams  with  high  tactical  ratings  (75%+)  showed 
significantly  higher  final  team  strength,  higher  correct  enemy 
classification  ratios,  and  better  average  resource 
effectiveness.  The  two  teams  with  the  lowest  final  team 
strength  (A  and  D)  had  poor  resource  effectiveness,  lower 
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average  identification  confidence  levels,  and  significantly 
more  vertical  upward  communications. 
e.  Analysis   of  Variance 

Table  9  summarizes  the  results  of  the  univariate 
analysis  of  variance  (ANOVA)  for  the  selected  dependent 
variables.  In  the  between  teams  analysis  all  of  the  selected 
dependant  variable  ANOVA' s  showed  a  significance  level  less 
than  0.05.  This  shows  that  significant  differences  existed 
between  each  of  the  teams'  performance,  strategy,  and 
communication  methods . 

2 .   Process  Analysis 

As  was  stated  before  in  Chapter  III  the  dependent 
variables  were  broken  into  three  categories;  Measures  of 
Performance,  Measures  of  Strategy,  and  Measures  of 
Communication.  Final  Team  Strength,  Number  of  False  Alarms, 
Number  of  Correct  Neutral  Identifications,  Correct  Enemy 
Classification  Ratio,  and  Average  Resource  Effectiveness  are 
all  measures  of  performance.  Average  Team  Latency,  Time  of 
First  Primary  Attack  Determination,  Average  Identification 
Confidence,  and  Average  number  of  ID  Actions  per  Task 
Identified  are  all  measures  of  strategy.  Total  Communication 
Rate,  Number  of  Messages  per  Target  Attacked,  Vertical 
Downward  Communication,  Vertical  Upward  Communication,  and 
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TABLE  8 


SUMMARY  STATISTICS 


BETWEEN  TEAMS  ANALYSIS 

MEANS 
STANDARD  DEVIATIONS 


TEAMS 

DVAR 

A 

B 

C 

D 

E 

F 

G 

1 

44.33 
22.07 

61.33 
26.81 

85.00 
22.75 

41.17 
19.46 

71.33 
14.05 

80.67 
6.95 

68.83 

17.11 

2 

4.83 
2.94 

4.67 
3.20 

3.33 
1.76 

4.50 
1.17 

4.17 
5.79 

5.00 
3.00 

9.33 
2.35 

5 

19.83 
2.94 

20.17 
3.15 

21.50 

1.76 

20.50 
0.82 

20.83 

5.58 

19.83 

3.00 

15.33 
2.46 

6 

0.84 
0.06 

0.76 
0.13 

0.92 
0.17 

0.67 
0.06 

0.94 
0.23 

0.93 
0.08 

0.57 
0.17 

24 

0.80 
0.13 

0.86 
0.11 

0.98 
0.07 

0.78 
0.07 

0.90 
0.06 

0.96 

0.06 

0.85 
0.08 

30 

375.03 
59.10 

338.02 
25.58 

260.29 
98.21 

303.25 
110.48 

131.28 
45.68 

307.56 
60.  49 

236.42 

71.60 

32 

879.83 
229. 74 

951.17 
141.47 

836.50 
119. 88 

708.83 
176.31 

956.33 
83.55 

1201.33 
167.12 

1200.50 
219.25 

38 

1.  64 
0.27 

1.97 
0.13 

2.03 
0.26 

1.73 
0.14 

2.11 
0.21 

1.96 
0.15 

1.98 
0.14 

39 

1.  66 
0.  70 

1.54 
0.30 

2.  92 
0.67 

2.36 
0.38 

3.78 

0.38 

3.43 
0.87 

3.07 
0.54 

40 

1.25 
0.24 

0.39 
0.28 

0.  65 
0.50 

0.71 
0.  61 

0.90 
0.39 

0.18 
0.14 

0.60 

0.27 

41 

3.04 
0.69 

1.10 
0.51 

1.38 
1.28 

1.53 
2.28 

2.51 
1.21 

0.49 
0.96 

1.80 
0.73 

44 

26.33 
1.51 

1.83 
3.33 

4.17 
12.83 

7.33 
17.95 

20.50 
3.02 

0.33 
12.20 

3.33 
7.88 

45 

9.33 

3.10 

2.33 
4.59 

4.83 
3.67 

7.33 

2.10 

3.50 

0.84 

0.67 
4.42 

0.33 
2.93 

46 

8.00 
6.57 

9.33 
8.33 

13.67 
7.82 

10.17 
2.40 

7.67 
11.62 

5.33 
6.31 

17.50 
4.83 

Horizontal  Communication  are  all  measures  of  communication. 
A  correlation  analysis  was  performed  using  the  correlation 
procedures  in  the  SAS  statistical  analysis  software  which 
produced  a  correlation  matrix  showing  interactions  between  the 
variables  (SAS  Institute  Incorporated,  1988)  . 


48 


TABLE  9 


ANOVA  -  BETWEEN  TEAMS  ANALYSIS 


DVAR 

SUM  OF  SQUARES 

F  VALUE 

Pr  >  F 

1 

10248.33 

6.42 

0.0001 

2 

135.23 

2.54 

0.0382 

5 

146.90 

2.94 

0.0199 

6 

0.74 

16.67 

0.0001 

24 

0.20 

7.15 

0.0001 

30 

228545. 90 

29.44 

0.0001 

32 

1205442.62 

8.55 

0.0001 

38 

1.03 

3.45 

0.0088 

39 

26.45 

107.54 

0.0001 

40 

4.29 

4.81 

0.0011 

41 

26.55 

3.13 

0.0146 

44 

3703.90 

7.72 

0.0001 

45 

406.90 

7.58 

0.0001 

46 

605.95 

2.54 

0.0382 

An  analysis  of  the  correlation  matrix  showed  a 
moderate  negative  correlation  (-0.35,  P<  0.0217)  showing  that 
increased  Vertical  Upward  Communications  leads  to  decreased 
Average  ID  Confidence.  Furthermore,  there  exists  a  moderate 
correlation  (0.42,  P<0.0057)  indicating  that  decreased  Average 
ID  Confidence  resulted  in  reduced  Average  Resource 
Effectiveness.  Additionally,  there  is  a  strong  correlation 
(0.82,  P<0.0001)  demonstrating  that  reduced  Average  Resource 
Effectiveness  caused  a  lower  Final  Team  Strength.  This 
analysis  is  supported  by  the  summary  statistics  of  the  between 
teams  analysis . 
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Further  analysis  of  the  correlation  matrix 
demonstrated  that  a  moderate  correlation  (0.36,  P<0.0197) 
existed  showing  that  an  increase  in  the  Average  Number  of  ID 
Actions  per  Task  Identified  led  to  an  increase  in  the  Average 
ID  Confidence  level.  Also,  a  there  was  a  moderate  correlation 
(0.42,  P<0.0057)  indicating  that  an  increase  in  the  Average  ID 
Confidence  level  caused  an  increase  in  the  Average  Resource 
Effectiveness.  As  indicated  before  increased  Average  Resource 
Effectiveness  results  in  higher  Final  Team  Strength.  This 
analysis  is  also  supported  by  the  summary  statistics  of  the 
between  teams  analysis . 

A  moderate  negative  correlation  (-0.41,  P<0.0073) 
indicates  a  decrease  in  the  Vertical  Upward  Communication  led 
to  an  increase  in  the  Average  Number  of  ID  Actions  per  Task 
Identified.  A  further  moderate  correlation  exists  (0.45, 
P<0.0027)  demonstrating  that  an  increase  in  the  Average  Number 
of  ID  Actions  per  Task  Identified  led  to  an  increase  in  the 
Final  Team  Strength. 
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V.   CONCLUSIONS  AND  FUTURE  RESEARCH 

A.  GOAL 

The  goal  of  this  study  was  to  determine  the  impact  of  (1) 
Neutral/Enemy   discriminability   uncertainty,    (2)      Team 
information  structure,   and   (3)    Leader's  involvement  in 
resource  coordination  upon  four  person  teams  consisting  of 
military  decisionmakers. 

B.  CONCLUSIONS  OF  STUDY 

Three  approaches  were  taken  in  the  analysis  of  the 
experimental  results;  between  cells  analysis,  between  teams 
analysis,  and  an  analysis  of  the  processes  that  affected  team 
performance.  The  between  cells  analysis  showed  that  of  the 
three  independent  variables  the  level  of  uncertainty  had  the 
most  significant  impact  on  team  performance.  With  an  active 
leader  there  was  an  increase  in  Horizontal  Communications, 
Average  ID  Actions  per  Task,  and  Time  of  First  Primary  Attack 
Determination  when  moving  from  a  decentralized  information 
structure  to  a  centralized  information  structure.  The 
opposite  was  true  when  the  leader  was  passive.  These 
variables  showed  a  decrease  when  moving  from  a  decentralized 
information  structure  to  a  centralized  information  structure. 
The  ANOVA  only  showed  three  dependent  variables  to  be 
significant;  Number  of  False  Alarms,  Number  of  Correct  Neutral 
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Identifications,  and  the  Average  ID  Confidence.  The  first  two 
of  these  are  very  closely  related  while  the  third  is  closely 
related  to  the  level  of  uncertainty. 

The  between  teams  analysis  shows  that  the  level  of 
tactical  training  within  a  team  directly  influenced  the 
performance  of  that  team.  Teams  with  a  more  tactical 
background  developed  a  more  coherent  team  strategy.  Average 
time-in-service  appeared  to  have  little  or  no  impact  on  team 
performance.  The  two  teams  with  the  lowest  Final  Team 
Strength  demonstrated  poor  resource  effectiveness,  lower 
confidence  levels,  and  significantly  more  upward 
communication . 

The  process  analysis  demonstrated  an  association  between 
communication  methods,  team  strategy,  and  team  performance. 
There  were  more  significant  correlations  between  strategy 
measures  and  performance  measures  than  communication  measures 
and  performance  measures,  although  the  communication  measures 
were  highly  correlated  to  each  other.  This  analysis  showed 
that  the  subordinates  who  tried  to  pass  a  large  amount  of 
information  up  the  chain  of  command  tended  to  assign  that 
information  much  lower  confidence  levels  causing  less 
effective  use  of  resources  by  the  team  and  subsequently  lower 
team  strength.  Conversely,  the  subordinates  who  passed  less 
information  up  the  chain-of-command  were  able  to  take  more 
sensor  readings  and  attained  a  higher  team  strength. 
Additionally,  teams  which  took  multiple  sensor  readings  on 
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unidentified  targets  were  able  to  assign  a  higher  confidence 
level  to  each  task  leading  to  better  resource  effectiveness 
and  finally  a  higher  final  team  strength. 

C.   LIMITATIONS  AND  FURTHER  RESEARCH 

There  were  five  primary  limitations  in  the  conduct  of  this 
experiment.  First,  communications  were  severely  limited. 
Each  decisionmaker  was  limited  to  a  pre-established  set  of 
messages  which  was  very  restrictive.  A  recommendation  would 
be  to  include  either  plain-text  messages  that  can  be  entered 
from  the  keyboard  or  to  establish  voice  communications . 

Second,  the  workspace  was  very  confined  and  the 
decisionmakers  were  not  adequately  separated  from  each  other. 
A  recommendation  would  be  to  ensure  physical  separation  of  all 
decisionmakers . 

Third,  during  the  conduct  of  the  experiment  it  was 
discovered  that  sometimes  the  leader  was  unable  to  engage 
targets  due  to  a  problem  in  the  software.  After  several 
discussions  with  the  primary  programmer  it  was  decided  that 
the  software  repair  would  have  to  wait  until  after  the 
conclusion  of  this  experiment.  This  led  to  removing  one  of 
the  dependent  variables  from  the  data  set  (DVAR  36  -  Number  of 
Direct  Attacks  by  Team  Leader) . 

Fourth,  the  hardware  that  was  used  for  this  experiment 
included  monochrome  monitors .  The  use  of  color  monitors  would 
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enhance  the  decisionmakers  ability  to  decipher  information 
from  the  screen. 

Fifth,  decisionmakers  were  unable  to  filter  the 
information  presented  on  the  screen.  The  ability  to  look  at 
only  targets  identified  as  enemy  or  neutral  would  reduce 
clutter  and  increase  readability  leading  to  less  confusion. 
This  is  especially  true  for  the  team  leader. 

Several  follow-on  studies  are  suggested  by  the  results 
provided  here.  First,  a  study  of  the  impact  of  implicit 
coordination,  as  a  strategy,  on  communications  and  team 
performance .  In  the  RAINCOAT  experiment  it  became  evident 
that  some  of  the  teams  developed  very  comprehensive  team 
strategies.  They  developed  rules  for  assigning  confidence 
levels  to  task  identifications,  resource  allocation  in  the 
overlap  areas  and,  types  of  communication  methods.  The  impact 
of  these  rules  showed  up  in  the  RAINCOAT  experiment  as 
improved  team  performance .  Future  studies  could  concentrate 
on  analyzing  this  implicit  coordination  as  an  independent 
variable  in  order  to  quantify  its  impact. 

Another  enhancement  to  this  study  would  be  to  move  the 
decisionmakers  into  a  more  tactical  setting  and  introduce 
external  noise  as  an  added  pressure.  The  experiment  was 
conducted  in  a  quiet  office  setting.  The  decisionmakers  were 
isolated  from  external  interference  and  even  prevented  from 
talking  to  each  other.  Future  studies  could  focus  on 
introducing  noise  into  the  environment  in  order  to  analyze  its 
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impact  on  the  decisionmaking  process.   This  would  move  the 
paradigm  closer  to  a  real  life  setting. 

Third,  subjective  workload  assessment  data  was  gathered 
for  this  experiment .  An  analysis  should  be  done  to  determine 
the  impact  of  perceived  workload  on  team  performance.  Data 
exists  for  each  decisionmaker  and  for  each  team. 
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